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Abstract: The Hirsch (2005) /i-index is now widely used as a metric to compare individual researchers. 
To evaluate it in the context of Australian Astronomy, the h- index for every member of the Astronomi- 
cal Society of Australia (ASA) is found using NASA's Astrophysics Data System Bibliographic Services 
(ADS). Percentiles of the ft-index distribution are detailed for a variety of categories of ASA members, 
including students. This enables a list of the top ten Australian researchers by /i-index to be pro- 
duced. These top researchers have /i-index values in the range 53 < h < 77, which is less than that 
recently reported for the American Astronomical Society Membership. We suggest that membership of 
extremely large consortia such as SDSS may partially explain the difference. We further suggest that 
many student ASA members with large h- index values have probably already received their Ph.D.'s 
and need to upgrade their ASA membership status. To attempt to specify the /i-index distribution 
relative to opportunity, we also detail the percentiles of its distribution by years since Ph.D. award 
date. This shows a steady increase in /i-index with seniority, as can be expected. 
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1 Introduction 

The modern research academic is judged as never be- 
fore: a large variety of metrics are now employed to 
determine the worth and merit of researchers, particu- 
larly when it comes to hiring. Anecdotally, one of the 
chief metrics used is the Hirsch index (/i-index; Hirsch 
2005). The h- index is formally defined as follows: "A 
scientist has index h if h of his or her N p papers have 
at least h citations each and the other (N p — h) pa- 
pers have < h citations each" (Hirsch 2005). Its mod- 
est simplicity is probably a prime factor in its rapid 
pick-up by major publishers (Ball 2005; Anon 2005). 
Moreover, this index is particularly useful as it has su- 
perior predictive power (in terms of productivity) for 
the future of researchers compared to the total number 
of career citations, career publications and mean cita- 
tions per paper (Hirsch 2007). Although other metrics 
and analyses exist (cf. Pearce 2004; Kurtz et al. 2005; 
Egghe 2006; Kosmulski 2006; Jin 2006; Blustin 2007; 
Jin et al. 2007; Bornmann, Mutz & Daniel 2008; Wu 
2010; Zyczkowski 2010), the /i-index remains as the 
most prominent of its class in the field. 

Recently, Conti et al. (2011) presented work on the 
Astronomer's H-R Diagram (number of Google search 
results versus citations and /i-index) for members of 
the American Astronomical Society (A AS). Contained 
within that presentation are a number interesting con- 
cepts: a top-ten list of /i-index of AAS members (span- 
ning the range 94 < h < 118) and the h indices of all 
AAS members. This work is motivated by the Conti et 
al. (2011) presentation and seeks to determine the typ- 
ical range of /i-index in Australian Astronomy which 
may be of use for future employers and employees in 
the community. The format of this work is as follows. 
In Section 2, we give an overview of the dataset that 
we use: the membership of the Astronomical Society 



of Australia. In Section 3 we determine percentiles of 
the /i-index distribution for a variety of ASA mem- 
bership categories, including students. To attempt to 
normalize relative to opportunity, we re-evaluate the 
ft-index distribution as a function time elapsed since 
Ph.D. award date in Section 4. Our conclusions are 
presented in Section 5. 

2 Data 

To determine the /i-index of Australian astronomers, 
we make use of the Astronomical Society of Australia 
(ASA) membership list. The membership list is a 
fair representation of the Australian astronomical com- 
munity: the majority of professional astronomers are 
members. Membership of the ASA comes in several 
difference categories which we use a single letter to 
abbreviate and detail in Table [T] The advantage of 
the ASA membership list is that we can differentiate 
different grades of members (i.e. amateurs from pro- 
fessional astronomers who actively publish) to better 
probe the /i-index in these sub-categories. 

For each ASA member, we then implement a search 
in NASA's Astrophysics Data System (ADS) to return 
a list of all refereed publications. We then sort this 
list according to citations to determine the /i-index for 
each ASA member. We note for prosperity that these 
searches were implemented on 24th-25th January 2011 
and were correct on a best-efforts basis as of said date 
range. 

A big issue in this methodology is attempting to 
tie down each individual to unique entries in ADS. Al- 
though the present author is blessed with a very rare 
surname, others in the community are not. For more 
common surnames, we use the first name and the mid- 
dle initials to help determine the /i-index of specific re- 
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Publications of the Astronomical Society of Australia 



Table 1: Categories of ASA membership, showing the number of members (N) in each category and 
the fraction of each that were excluded (X) from the subsequent analysis due to name confusion (see 
Section 2). We ignore the extra categories of Associate Society member (i.e. persons who are members 
of other learned societies that are likely to not possess Ph.D.'s in astronomy) and corporate members. It 
is important to note that individual researchers can belong to multiple categories; e.g. retired, overseas 
fellows. 



Category 


N 


X 


Notes 


M 


262 


0.10 


Member: Full professional member with a Ph.D. in astronomy or related discipline. 


F 


81 


0.10 


Fellow: Senior members of the community with potentially decades of experience. 


S 


164 


0.09 


Student: Post-graduate students studying toward Ph.D.'s in astronomy. 


H 


15 


0.00 


Honourary: Elected by the ASA council for distinguished contributions. 


R 


54 


0.11 


Retired. 





56 


0.10 


Overseas. 


A 


14 


0.00 


Associate Members: 








Educators, communicators and amateur astronomers lacking a Ph.D. 



searchers, including attempting common substitutions 
for first names; e.g., 'Bill' for 'William', etc. However, 
for the very common surnames (e.g. Smith), this is 
not always possible. Therefore, the subsequent analy- 
sis in this work does not include any names for which 
we could not adequately differentiate a single individ- 
ual in the literature in a reasonable amount of time. 
This affects ~10% of the membership list and is la- 
belled X in Table [T] We caution that the subsequent 
analysis should therefore be regarded as incomplete: 
the inclusion of these names could increase or decrease 
the relative rankings of individuals within the ASA 
membership. We also note that we make no attempt 
to exclude self-citations in our analysis (e.g. Pimbblet 
2011). Finally, it may be the case that some of the 
categories may not be up-to-date due to (e.g.) student 
members gaining their Ph.D.'s and either not upgrad- 
ing to full membership status immediately or the list 
itself not being updated immediately. 



3 /i-index by ASA Member- 
ship Category 

This simplest point of departure for the /i-index anal- 
ysis is to pull out the top ten: those people who could 
rightly be called academic giants in their own right in 
the community. To do this, we simply rank all profes- 
sional members who are not based overseas (i.e. cat- 
egories M+F+S+H+R; Table [TJ. This top ten list is 
presented in Table [2] 

Although we refrain from commenting on individ- 
uals in this list, it is instructive to compare it to Conti 
et al.'s (2011) list. The fo-index values for the top 
ten AAS members is much higher than the Australian 
(94 < h < 118 versus 53 < h < 77). Examination of 
Conti et al.'s (2011) figures suggests that membership 
of very large observational programmes such as the 
Sloan Digital Sky Survey (SDSS; e.g. Abazajian et al. 
2009) can boost researcher's ft-index above mean val- 
ues. It is certainly the case that the Australian top ten 




Figure 1: Histogram of /i-index for all ASA mem- 
bers, excluding those for whom name confusion 
could not be resolved. 
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is dominated by non-SDSS professionals and we there- 
fore suggest that most of the difference seen between 
the two samples could be due to this effect. Indeed, 
seven of the AAS's top ten (Conti et al. 20fl) are con- 
tained in the author list of Abazajian et al. (2009). 
However, we do note that the Australian top ten does 
contain a number of members of other consortia (not 
as large or extensive as SDSS) such as the 2dF Galaxy 
Redshift Survey (e.g. Colless et al. 2001). Moreover, 
the majority of the listed researchers in Table [5] also 
feature in Thomson Reuters' ISI's highly cited list (isi- 
highlycited.com) for space science. 



Table 2: Top 10 ft-index for Australian Astronomy, 



excluding overseas professionals. 

Rank Name h 

~=i Ken FREEMAN 77 

= 1 Jeremy MOULD 77 

3 Karl GLAZEBROOK 71 

4 Dick MANCHESTER 68 

5 Michael DOPITA 64 

6 Warrick COUCH 61 

7 Matthew COLLESS 56 

8 Brian SCHMIDT 54 
=9 Mike BESSELL 53 



=9 Joss BLAND-HAWTHORN 53 



But what of the rest of the community? In Figure[T] 
we display a histogram of ft-index for all ASA mem- 
bers. This graph is dominated by those members hav- 
ing a zero ft-index or slightly above, much as the AAS 
community is (Conti et al. 2011). The vast majority of 
these members are student members, many of whom 
are likely to not have published. Even if they have 
published, the duration of the Ph.D. may mean that 
sufficient time has not elapsed to gain large numbers 
of citations and that only the very exceptional papers 
produced by students garner large number of citations 
immediately. Clearly students in present-day collabo- 
rations such as WiggleZ (Drinkwater et al. 2010) will 
benefit from this effect much in the same way as SDSS 
members receive a boost. 

To analyse the content of Figure [TJ in a more in- 
depth manner, we now create sub-samples of the ASA 
membership according to grade and determine vari- 
ous percentiles of the ft-index distribution. These per- 
centiles are presented in Table [3] We do not present 
results for individual categories H, O, R and A due to 
low numbers. This can be seen in the relatively tiny 
difference between the percentiles quoted for M+F+R- 
O versus M+F-R-O samples in Table 

We start by discussing the student membership re- 
sult. At the upper echelons, students appear to have 
an ft-index comparable of junior professionals. But a 
careful analysis of the membership list reveals that this 
is exactly what these students are: junior profession- 
als who should be in the M category. We argue that 
anything above the 90th percentile for the S category 
should be regarded with suspicion. 



Naturally, the fellows occupy much higher ft-index 
values than the regular members do. The effect of 
adding or removing the retirees from the M+F sample 
is modest: the most noticeable effect is at the upper 
echelons of the scale. However, the major problem of 
this analysis is that it does not specify the ft-index 
relative to opportunity. To remedy this, we now try 
to divide up the ASA membership according to years 
since the award of a Ph.D. 

4 h- index by Years Since Ph.D. 
Award 

Even the award year of a Ph.D. must be regarded with 
healthy suspicion as a metric for performance relative 
to opportunity. This is especially true for early-career 
researchers who may complete their Ph.D. whilst un- 
dertaking their first post-doctoral placement and for 
the many researchers who have had significant time 
away from the profession; the present author included. 

To determine the award date of the Ph.D., we use 
the results from ADS where available. If the Ph.D. 
is not listed in ADS, then we use the date of the sec- 
ond first-author refereed publication by the member 
as a compromise proxy for this date, given the dis- 
tribution of the S sample in Table [3] This date was 
determined for all ASA members in the M+F+R-O 
category. Where no date could be determined by ei- 
ther method, the member was simply removed from 
the list. This may have the effect of meaning that the 
percentiles for this sample are upper limits as we have 
missed doctoral researchers who have few first author 
publications. We present the percentiles of this dis- 
tribution in Table [4] The results show a fairly steady 
progression as one increases in seniority from Ph.D. 
award date without any obvious discrepancies, as may 
be expected. However, one comment to be made is 
that there seems many less young professionals in the 
samples than there perhaps should be (given the num- 
bers in more senior years). This tentatively suggests 
that new recruits to Australian Astronomy may not be 
joining the ASA immediately. 

Further, not all areas and sub-disciplines of sci- 
ence and astronomy may be equal. Those researchers 
involved in (e.g.) instrumentation may have a very dif- 
ferent ft-index distribution to those researching obser- 
vational cosmology (particularly those in larger-sized 
consortia). 

5 Conclusions 

This work has presented an analysis of the ft-index dis- 
tributions for present members of the ASA. As well as 
deriving a top ten (Table [2]), we have presented the 
percentiles for various sub-samples of the ASA's mem- 
bership, including student statistics (Table [3]). Wc 
have also attempted to analyse the distribution rela- 
tive to opportunity by detailing the percentiles by time 
elapsed since Ph.D. award date (Table [4]). 

Clearly the ft-index is a crude estimator of the 
value of a researcher and should not be used in iso- 
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Publications of the Astronomical Society of Australia 



Table 3: Percentiles of /i-index distribution by ASA membership grade. N gives the total number of 

members for eac h row. 

Category N Percentile 







25 


50 


75 


90 


95 


97.5 


99 


M 


235 


6.0 


12.0 


21.0 


32.5 


43.0 


47.8 


52.4 


F 


73 


21.0 


30.0 


39.8 


50.0 


59.2 


69.8 


75.4 


S 


149 


0.0 


0.0 


2.0 


4.0 


5.0 


6.7 


9.5 


M+F-R-0 


227 


7.0 


15.0 


28.0 


41.0 


45.0 


53.0 


63.7 


M+F+R-0 


266 


6.0 


15.0 


28.0 


40.0 


45.4 


53.3 


68.8 


M+F+S-R-0 


374 


1.0 


5.0 


19.0 


33.6 


42.3 


46.7 


59.7 



Table 4: Percentiles of ft,-index distribution by years since Ph.D. award (explicitly: 2011 minus the award 
date). Only members in M+R+F-0 categories are included. N gives the total number of members for 
each row. 



Years since 


N 






Percentile 






Ph.D. award 




25 


50 


75 


90 


95 


97.5 


99 


0-5 


32 


5.0 


6.0 


9.0 


10.0 


13.2 


15.8 


16.3 


6-10 


42 


8.5 


14.0 


19.0 


22.0 


25.0 


27.8 


29.7 


11-20 


74 


12.5 


19.0 


28.5 


39.6 


43.4 


53.1 


58.4 


>20 


94 


12.0 


25.0 


38.0 


47.2 


55.1 


62.9 


77.0 



lation to other metrics, even if it is a good predictor of 
future productivity (Hirsch 2007). It will be instruc- 
tive to re-visit this analysis in future years or decades 
to determine how the field has changed. 

We terminate this work with a caveat emptor: there 
are known deficiencies in this analysis such as numer- 
ous missing persons (who are not ASA members) whose 
statistics may alter the results presented. We have 
tried to be up-front with various caveats throughout 
this work, but there may yet be unknown unknowns 
present as well. Further, there may exist transcription 
errors that went un-detected during the data assem- 
bly stage. However, as far as possible, we believe the 
numbers quoted in this work are accurate. 
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